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ABSTRACT 

Observation competence plays a fundamental role in outdoor scientific investigation. The computerized concept 
mapping technique as a Mindtool has shown the potential for enhancing meaningful learning in science 
education. The purposes of the present study are to develop a concept map integrated mobile learning design for 
ecology observation and to examine the implementation effect for elementary school children. Also, the 
difference in growth slopes between gifted and average students is investigated. Eighteen gifted students and 
thirty average students were included in this study. A computerized dynamic assessment system which 
combines a Computerized Ecology Observation Competence Assessment (CEOCA) and a concept map 
integrated ecology observation learning design were implemented. The results of the hierarchical linear model 
(HLM) analysis reveal that the overall growth slope is significant (p=0.27, p<.01). The effect size is 0.53. The 
growth slope of the gifted students is a little higher than that of the average students. However, the slope 
difference between ability groups is not significant. The results suggest that a well designed concept map 
integrated learning system demonstrates very promising potential for enhancing both the gifted and average 
students’ mobile observation competence. The system developed in this study could be a useful resource for 
elementary school outdoor learning design. 

Keywords: concept map, dynamic assessment, observation competence, mobile learning, growth slope 

BACKGROUND AND OBJECTIVE 

The issue of ecological environmental protection becomes increasingly crucial for environmental education. 
Regrettably, passion in environment inquiry is rarely fostered in the majority of classrooms despite the clear 
interest in many children, especially those in the primary years in science. Science literacy is the creative 
exploration of meaning in the natural world. It requires both observing and thinking. Scientific observation is 
the process of gathering information about objects, events or processes in a careful, orderly way. Organisms are 
characterized by diversity and unity. Despite the diversity, biologists are able to group organisms based on 
shared similarities. Classification helps us understand diversity. Life has many levels of organization. Each level 
of organization is more complex and has more properties than the previous level. In other words, concept of 
hierarchy is important for ecology observation. Observation generally involves using the senses, particularly 
sight and hearing. Observations may lead to unanswered questions. Scientific discovery often takes place when 
a scientist observes something no one has noticed before (Hung, Lin, & Hwang, 2010). 

Outdoor teaching is widely recognized as one of the best alternative teaching methods for scientific observation 
learning. However, some outdoor teaching approaches are ineffective because students lack expert guidance and 
appropriate outdoor learning tools. With the advantages of portability and easy information access, the use of 
mobile technology is a growing trend in education. Therefore, the application of information technology in 
outdoor teaching has become an attractive research topic (Hung, Lin & Hwang, 2009). Outdoor teaching 
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provides an excellent opportunity for scientific observation and inquiry learning. The success of environmental 
science education is not only dependent on the knowledge and understanding of environmental challenges, but 
also on impassioned participation in environmental conservation activities. Observation competence plays a 
fundamental role in outdoor scientific investigation. Through well-organized observation activities on field trips, 
students became more motivated and engaged in science learning to enhance problem-solving competence and 
meaningful learning. However, the key point of effective teaching depends on validated instruction design. 

In a conventional learning activity for outdoor teaching, the students are guided by a learning-mission sheet 
prepared by the teacher, and write down their findings on the sheet after visiting each of the learning objects. 
Such a learning activity allows the students to observe the real-world objects without personalized guidance or 
support; consequently, some students might fail to pay attention to the key features to be observed, or fail to 
complete the mission owing to a lack of sufficient information or guidance (Chu et al., 2008; Hwang, Kuo, Yin, 
& Chuang, 2010). 

The rapid progress and advances in innovative mobile, wireless and sensor technologies have substantially 
revolutionized the ways in which outdoor learning activities for science education can be carried out (Hwang, 
Tsai, & Yang, 2008: Chu, Hwang, & Tsai, 2010). Mobile devices, e.g. mobile phones and Personal Digital 
Assistants (PDAs), have also become more popular as cognitive tools in science learning (Hwang, Yang, Tsai, 
& Yang, 2009; Hwang, Kuo, Yin, & Chuang, 2010; Vogel, Spikol, Kurti, & Milrad, 2010). Due to the attractive 
features of handheld computers such as portability, adaptability, flexibility, intuitiveness, and comparatively 
cheap prices, ubiquitous/mobile learning which integrates handheld computers with wireless networks in 
teaching and learning has become one of the leading topics in educational research (Chen, Hwang, Yang, 
Chend, & Huang, 2009; Hwang, Kuo, Yin, & Chuang, 2010; Liu & Hwang, 2010; Shih, Chuang, & Hwang, 
2010; Hwang & Chang, 2011). Especially, with the advantages of portability and easy information access, 
mobile technologies are now used frequently in outdoor scientific investigation activities. Several studies have 
been conducted to investigate the effectiveness of mobile and ubiquitous learning which can provide an 
opportunity for students to keep accessing digital resources while learning in real-world scenarios (Chen et al, 
2003; Westerlund, 2008; Chu, Hwang, & Tsai, 2010). 

Although those new technologies seem to be promising, researchers have pointed out that the students’ learning 
achievements could be disappointing without the aid of effective learning strategies or tools to engage them in 
improving their knowledge structure (Chu, Hwang, Tsai, & Tseng, 2010; Hwang, Shi, & Chu, in press), which 
is regarded as an important component of understanding in a subject domain, especially in science (Novak, 
1990). The knowledge structure of experts and successful learners is characterized by elaborate, highly 
integrated frameworks of related concepts (Mintzes, Wandersee, & Novak, 1997). A knowledge structure, then, 
might well be considered as an important but generally unmeasured aspect of science achievement. In order to 
understand the advanced scientific concepts of the various disciplines, students cannot rely on the simple 
memorization of facts or the enrichment of their naive, intuitive theories. They need to be able to restructure 
their prior knowledge which is based on everyday experience and lay culture. The restructuring is known as 
conceptual change (Vosniadou, 2007). Concept mapping techniques are interpreted as representative of students’ 
knowledge structures and so might provide one possible means of tapping into a student’s conceptual 
knowledge structure (Mintzes, Wandersee, & Novak, 1997). A concept map is a graph structure containing 
nodes that are interlinked by labelled, directed arcs. Concept maps can be used as a knowledge representation 
tool to reflect relationships that exist between concepts that reside within an individual’s long-term memory. 
When constructing a concept map, the focus is the relationships among concepts. The combination of two 
concepts connected by a linking line and labeled by a linking word creates a proposition, which is the smallest 
linguistic unit that carries meaning (Jacobs-Lawson & Hershey, 2002). 

Generally speaking, in the practice of concept mapping, weighted scores are assigned based on the organization 
of a map’s hierarchical structure, concept-links, and cross-links. Especially, the inclusion of cross-links is a 
significant characteristic of concept maps in representing creative thinking. Cross-links are important to show 
that the learner understands the conceptual relationships between the sub-domains in the map (Novak & Canas, 
2008). The concept mapping tasks are categorized as either “fill-in the blank” tasks where learners are provided 
with a blank structure of a map and lists of concepts and linking phrases, or “construct a map” tasks where 
learners are free to make their own choices. Assessments based on “construct a map” tasks more accurately 
evaluate differences in learners’ knowledge structures and elicit more high-order cognitive processes (Anohina, 
Graudina, & Grundspenkis, 2007). “Construct a map” tasks are better than “fill-in the blank” tasks for capturing 
students’ partial knowledge. However, “fill-in the blank” tasks can be scored more efficiently than “construct a 
map” tasks. Based on their characteristics, if used as an assessment tool, “construct a map” tasks are more 
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suitable for formative assessment, while “fill-in the blank” tasks are a better fit for large scale assessment (Yin 
et al., 2005). 

The inclusion of concept mapping in learning activities is also recognized as one way to summarize 
understandings acquired by students after they study instructional materials. Both summarization writing and 
construction of concept maps are appropriate ways to improve students’ knowledge growth. They share a 
substantial common cognitive process in correspondence with concept mapping to select concepts, make 
propositions, and to hierarchize and structuralize the key concepts. It has also been found that training in 
concept mapping should be beneficial for enhancing text comprehension, summarization and writing skills 
(Chang, Sung & Chen, 2002; Riley & Ahlberg, 2004). Moreover, concept mapping creates less cognitive load 
than summarization and essay writing. However, when constructing concept maps in the conventional 
pencil-paper way, students could need extra effort in deletion and revision of partial or even whole maps 
(Anderson-Inman & Ditson, 1999). Researchers (Chang, Chen, & Sung 2001) have identified some weaknesses 
which make traditional style concept mapping inconvenient for interaction and feedback between learners and 
instructors. In contrast, computerized concept mapping assessment provides a number of advantages: greater 
flexibility of revision, instant feedback to learners, extensive feedback to teachers, reduced errors in comparison 
with human marking, as well as decreased time needed for supervising and marking of assessments (Akkaya, 
Karakirik, & Durmus, 2005; Tsai et al., 2001). 

The purposes of this study are to develop a concept map integrated mobile learning system for ecology 
observation and to examine the implementation effects for elementary school children. The difference in 
learning growth between gifted and average students is also investigated. 

CONCEPT-MAP INTEGRATED DYNAMIC ASSESSMENT SYSTEM 

The Concept-map Integrated Dynamic Assessment System (CIDAS) combines the Computerized Ecology 
Observation Competence Assessment (CEOCA) and a concept map integrated ecology observation learning 
design. The concept-map is applied as a Mindtool to enhance students’ ecology observation competence. The 
purpose of developing CIDAS is to investigate the characteristics of students’ learning progress and the need for 
supportive feedback for students’ knowledge construction. The system includes a worksheet-embedded PDA 
learning design, an e-library, and online feedback as scaffolding to guide, clarify, stimulate, and monitor 
students’ observational inquiries. The CIDAS functions as both an instruction and an assessment tool. The 
online feedback and the e-library have been developed to facilitate effective learning. 

The Computerized Ecology Observation Competence Assessment (CEOCA) is developed for assessing 
students’ ecology observation competence. The test items are presented with real pictures, films or concept 
maps. The CEOCA consists of three facets, i.e. knowledge, observation and conceptual relationships. Totally 
there are 40 items in CEOCA. Table 1 shows the test specifications for CEOCA. The alpha coefficient of 
CEOCA is around 0.72. The correlations between CEOCA and school grades, i.e. science, mathematics and 
Chinese, are 0.54, 0.47 and 0.51, respectively. The CEOCA demonstrates moderate convergent and discriminant 
validity (Hung, Hwang, Lin, Hung, & Wu, 2010). Figure 1 provides an illustrative example of concept map task. 
The task is to measure students’ competence on feature identifications of ecology system. Students can drag an 
alternative from right hand side to fill in the blanks of a structured concept map. The item format is also function 
as a model of knowledge structure. 


Table 1. Test specifications for CEOCA 


Content 

Basic 

Proficient 

Advanced 

Total 

Knowledge 

2 

12 

6 

20 

Observation 

5 

6 

3 

14 

Relationship 

5 

1 

0 

6 

Total 

12 

19 

9 

40 
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Drag a proper concept from the right side into the blank. 



Figure 1. An illustration of concept map item on CEOCA 


The concept map integrated ecology observation system was designed based on the three-layer framework of the 
cognitive load-oriented ecology observation approach proposed by Hung, Lin and Hwang (2010). The concept 
map was integrated into this system to scaffold students’ knowledge structure. The learning tasks are transferred 
by wireless communication to students’ handheld PDAs in the field. The students’ responses to tasks and their 
observation records are also sent back to the learning system. In the learning system, the PDAs function as 
portable notebooks and walking encyclopedias. The learning system also provides guided tasks, immediate 
feedback, e-library search functions for mangrove wetland ecological systems, and an e-diary editor. Figure 2 
presents the illustration of these supporting functions. The concept map tasks are embedded in the learning 
system to scaffold students’ ecology conceptual schema. The tasks in activity I present a structured one-level 
concept map to clarify students’ basic knowledge about the features and classifications of mangrove species. 
The tasks in activity II especially encourage students to take notes on two-level comparisons of different species. 
The tasks in activity III guide students to use what they have learned to draw three-level inferences about the 
relationships among species. 



Figure 2. The illustration of the supporting functions on PDA 


A preliminary e-library consisting of a database of various species in mangrove wetland ecological system was 
developed for CIDAS. The content of the e-library is divided into two parts, i.e. plants (e.g. main plants and 
associated plants) and animals (e.g. birds, crabs, fish, and shellfish). Table 2 provides illustrative examples of 
e-library entries for animals. 
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Table 2. An illustrative example of e-library entries for animals 


No. specie 

behavior 

feature 

habitat 

special note 

DAAI Black-faced 
B001 Spoonbill 

winter 

migratory 

The length of the body is 74 - 85cm. 
The face and the beak are black. The 
end of the beak is wide and flat like a 
spoon. The feathers are mostly white. 
The wingspan is about 130-142 cm. 
Legs are also black. 

fish farm, 
swamp 

More than 1,000 
birds immigrated 
to Chiku wetland 
this year. 


METHOD 

Figure 3 presents the experiment design for evaluating the effects of using CIDAS on the students’ ecology 
observation competencies. The CEOCA is administered three times to generate the growth slope for participants. 
The growth slope of CEOCA is the criterion variable for examining the intervention effect of CIDAS. The tasks 
included in the worksheets are multiple-choice, short-answer, constructed responses and concept-map. The 
adaptive feedback is provided online to support the learning progress of CIDAS. 


December, 2009 


26 th December, 2009 
to 

16 th January, 2010 


Middle of January, 
2010 


12 th to 28 th 
January, 2010 


March, 2010 



Figure 3. The CIDAS implementation flowchart 


Participants and Procedures 

Forty-eight fifth to sixth-grade students from six schools in Taiwan participated in this study. Eighteen 
participants were gifted students recruited from gifted classes while the other 30 were average students recruited 
from regular classes. Three mangrove wetland field observation trips were arranged within 3 months. During the 
field trips, each of the participants was equipped with a PDA, a digital camera, and a telescope. The observation 
learning objectives (Table 3) were mostly guided by the worksheets. On the first trip, the students were guided 
to carefully observe some target items. On the second trip, they were suggested to focus on comparison tasks. 
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On the last trip, they were encouraged to draw the links among the animals and plants they observed. The 
participants used the equipment to record what they observed, and completed the worksheet tasks using the 
PDAs in the field. After each field trip, the participants were asked to finish learning diaries on the websites, 
based on their observations, within a week. Due to a lack of equipment, participants were divided into two 
experimental groups. The gifted and average students were mixed in these two groups. 

For each task completed on PDA, score or feedback will be provided immediately. Students are expected to 
conduct a scientific inquiry based on their observation records on PDAs after each field trip. They carried out 
further in-depth inquiry based upon the notes and questions raised autonomously during the field observations. 
The automatic scoring feedback system is developed to motivate students to elaborate their records, reflections 
and inquiries. 


Table 3. The descriptions of three outdoor activities for CIDAS 


Activity 

Site 

Time 

Objectives 

Tasks 

i st 

Chiku Wetland 

6 hr 

Observation on target items 

Multiple choice 

Short answer 
Concept-map 

2 nd 

Mangrove Pond 

3 hr 

Comparisons among targets 

Multiple choice 

Multiple true-false 
Constructed responses 
Concept-map 

3 rd 

Sihcao Wetland 

3 hr 

Relations of Ecosystem 

Multiple-choice 

Multiple true-false Constructed 
responses 

Concept-map 


Dynamic assessment instruments 

The concept map integrated ecology observation worksheet design was revised from the three-layer framework 
of observation worksheet design (Hung, Lin, & Hwang, 2010). Based on the guided inquiry learning theory and 
concept mapping theory, tasks are developed to sequentially scaffold students’ knowledge construction (Hung, 
Hwang, & Hung, 2010). The worksheet design arranged the learning objectives into three layers to balance the 
students’ cognitive load and challenge step by step. The three layers consisted of a) guided observation with 
one-level concept map tasks, b) autonomous observation with two-level concept map tasks, and c) scientific 
inquiry with three-level concept map tasks. Table 4 provides the specification table for the ecology observation 
formative assessment. The scoring rubrics for these concept map tasks have also been proposed and validated by 
Hung, Hwang and Hung (2010). 


Table 4, The specification table for ecology observation formative assessment 


Item type 

Feature and 
classification of species 

Feature contrast 
of species 

Conceptual connection 
and extension 

Total 

Multiple choice 

25 

11 

7 

43 

Multiple true-false 

0 

4 

3 

7 

Short-answer 

3 

3 

3 

9 

Concept map 

2 

2 

2 

6 

Total 

30 

20 

15 

65 


FINDINGS AND DISCUSSION 

The ecology observation learning potential is defined as the slope of students’ performance on CEOCA in three 
month intervals. The hierarchical linear model (HLM) is applied for this analysis. Table 5 presents the 
descriptive statistics on three CEOCA z-scores for all students. For pre-test, the subjects included perform very 
close to the norm (0.04 vs. 0.00) of 5 th and 6 th graders with a smaller standard deviation . vs. 1.00). The results 
of student 2 nd -test demonstrate substantial improvement on CEOCA. The follow-up test also suggests the 
subjects can maintain what they have learned quite well. In other words, after CIDAS intervention, students 
show persistent better observation competence. Table 6 provides the results of the students’ growth slope on the 
CEOCA. The results demonstrate that the overall growth slope is significant (y/=0.27,p<.01). In other words, for 
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all the students included in this study, the CIDAS is effective for promoting their ecology observation 
competence. The effect size is 0.53 (about one half standard deviation, see Table 5). Relatively speaking, the 
effect size is noticeable compared to the difference between grades, namely 0.04 (Hung, Hwang, & Lin, 2010). 


Table 5. Descriptive statistics on three CEOCA z-scores for gifted and average students 



Gifted ( N= 

=18) 


Average ( N=30 ) 


Total (N=48) 


Mean 

SD 


Mean SD 

Mean 

SD 

pre-test 

0.27 

0.51 


-0.10 0.55 

0.04 

0.56 

2 nd -test 

0.76 

0.68 


0.47 0.65 

0.58 

0.67 

follow-up test 

0.98 

0.71 


0.33 0.85 

0.57 

0.85 

Table 6. Final estimation of fixed effects of unconditional model on 

CEOCA slope (N=48) 

Fixed Effect 

Coefficient 

Std. error 

T-ratio 

df 

Sig. 

INTRCPT, 

B0 

0.13 


0.08 

1.67 

47 

0.10 

TIME slope, 
B1 

0.27 


0.06 

4.69 

47 

0.00 


The contrast of the growth slopes for the two different ability groups on the CEOCA is provided in Figure 4 and 
Table 7. The growth slope of the gifted students is a little higher than that of the average students (about 0.14). 
However, the slope difference is not significant. Some concept mapping assessment has suggested that the 
concept maps constructed by high performing students are qualitatively and quantitatively superior to those of 
average performing students in science (Austin & Shore, 1993). Although gifted students inherently 
possess higher academic achievement, their scientific observation competence progress slopes may not 
significantly surpass those of the average students in a well-designed mobile learning system. However, the 
results of Table 7 also imply that the reservation effect between ability groups might be an important issue for 
further investigation. It seems that the average students forget some of the concepts or knowledge in follow-up 
test. On the other hand, the gifted students may keep on internalized what have been learned after the 
intervention. 


1.2 

1 

^ 0.8 

« 0.6 

< 

8 04 
O 0.2 

0 

- 0.2 


Figure 4. Contrast of the growth profiles on CEOCA z-score for gifted and average students 


Table 7. Final estimation of fixed effects on CEOCA slopes for gifted and average students (N=48) 


Fixed Effect 

Coefficient 

Std.error 

T-ratio 

Df 

Sig. 

INTRCPT, 

Average student 

0.02 

0.10 

0.17 

46 

0.86 

B0 

Gifted student 

0.30 

0.15 

1.94 

46 

0.06 

TIME slope, 

Average student 

0.22 

0.07 

3.00 

46 

0.00 

B1 

Gifted student 

0.14 

0.12 

1.20 

46 

0.24 


CONCLUSION 
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Scientific observation learning occurs through the learner’s participation in a “community of practice”. In other 
words, learners participate in complex, real world challenges. Outdoor observations are usually fun but 
superficial for most elementary school students. Any innovative instruction that ignores the structures which 
constitute human cognitive architecture is not likely to be effective. Novice observers should be provided with 
effective guidance on the concepts and procedures required for observations. To help learners understand the 
structure of what is to be learned is very crucial. Understanding leads to increased awareness and improved 
ability to remember. The interconnection of design of activity systems and learning cognition is very important. 
Formative assessment refers to assessment that is specifically intended to generate feedback on performance to 
improve and accelerate learning (Sadler, 1998). Black and William (1998) also claimed that well designed 
formative assessments have a great positive effect on students’ learning. Online feedback given as part of 
formative assessment helps learners to focus on and achieve their leaning goals in field. In other words, concept 
map integrated formative assessment of mobile learning plays a supportive role for novice learners. 

Ecology observation activity will be much more effective if mobile learning and assessment technologies could 
be appropriately integrated for elementary school students. In this study, a dynamic assessment approach is 
adopted to develop an ecology observation learning system for mobile learning. Moreover, concept map is 
integrated into three mobile observation activities to promote students’ ecology observation competences. 
Students are guided to observe the critical features of targeted species, to compare the similar species and to 
relate the species to the environment sequentially. Students are also encouraged to take notes and to pursue their 
own learning goal besides responding to the worksheet tasks. Online scoring and feedback are provided to 
support and monitor students’ learning progress. The results of this study suggest that the concept map 
integrated mobile learning design successfully enhances observation competencies for both average and gifted 
students. It is also found that the gifted students’ scientific observation performance is superior to that of the 
average students through all learning stages. However, there is no significant difference in the growth slope 
between the gifted and the average students. This suggests that the concept map integrated mobile learning 
demonstrates very promising potential for all students. Generally speaking, the intervention effect of CIDAS is 
quite substantial for both the average and gifted students. Without any extra effort, the difference of observation 
competence between the 5 th and 6 th graders is minor (Hung, Hwang, & Lin, 2010). In other words, a well 
designed instructional innovation can be beneficial for all students. The preliminary results suggest CIDAS is a 
valuable resource for mainstream science education. 

The structured concept map tasks are applied to demonstrate an effective way of knowledge organization. 
Students provide more scientific vocabularies, comparison and relational statements gradually in their diaries. 
The structural changes of students’ knowledge are also worth detailed discussions to reveal the characteristics of 
different growth profiles. So, using the formative assessment results (such as worksheet or diary) to crossed 
validate the inferences made by objective tests will be a promising direction for further studies. Moreover, the 
larger gap on the follow-up test on CEOCA between the gifted and the average students reveals an important 
issue for further investigations on the reservation effect for different ability groups. The differential effect of 
concept map embedded learning design also needs larger sample and longer intervention to clarify. 
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